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Caution—Convenience Sample Ahead! 


By Tom Treiman, Heather Scroggins, and Michele Baumer 


Information Needs: MDC managers and decision 
makers often need information on what “the public” 
thinks about a resource management issue—from how 
users like a new trail or exhibit to how the state should 


manage the deer population. For any public survey, 
we must identify the target audience and how results 
will be used. 


Two Horse Rider Surveys 

In 2008, MDC developed a printed survey asking horse 
riders how often they rode on public land, where they 
rode, how much they spent, and additional questions 
about their attitudes regarding equestrian trails. The 
survey was printed in two versions. One was sent to a 
randomly selected sample of riders, the other was a 
convenience sample distributed to saddle clubs, trail 
rides, and other interested parties. The Random 
Sample was to provide unbiased statewide 
information, while the Convenience Sample was to 
meet the desire of the riding public to provide input. 


A convenience sample is much more likely to be 
completed by avid users, who have more opportunities 
to see it, and by users with stronger opinions about the 
issue addressed, who are more motivated. A random 
sample should statistically represent the whole 
population from which the sample is drawn, in this case 
all Missouri landowners with horses. With a 
convenience sample, it may never be known how 
many people received it, only how many were sent in. 
By contrast, with a well-constructed random sample, 
which includes a database of names and addresses 
and numbered surveys, the response rate is known. 


Horse Rider Results 

For the Random Survey the response rate was 48.1% 
(2,286/4,750). Over 1,000 Convenience Surveys were 
sent by request to saddle clubs, trail riders, and 
individuals and 510 replies were completed and 
returned. 


There were 13 numerical questions on the surveys 
(e.g., “How many times did you ride last year?”) and 
78 “text” questions (e.g., “Mountain bikers cause 
problems for horses. Do you agree or disagree?”). Of 
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these 91 items, responses for 50 are statistically 
different between the Convenience and Random 
surveys showing the two populations are different. 


More importantly, some of those differences were 
substantive; they really mattered (See Fig. 1)! Using 
the random sample, we estimated that Missouri horse 
owners rode public trails about 180,000 times in 2007. 
Basing that estimate on the average number of times 
ridden by the convenience sample riders would have 
yielded a very different (higher) number. Basing a 
decision about a campground on the convenience 
sample versus the random sample also would yield 
very different decisions. 





Question Convenience | Random 





How many times did you ride on public lands in 2007? 





How many times did you ride the 1“ [favorite] area? 
How long was your average trip to the Ist area? 


Number\length of trails on area is “very important”. 69.9% 49.4% 





Availability of camping is “very important”. 45.9% 35.8% 





Single track is the ideal trail width. 29.2% 16.9% 

Strongly agree that riders obey trail rules. 
Would be willing to volunteer. 
Member of a saddle club or other horse group. 





Fig. 1. Survey questions with large differences 
between Convenience and Random surveys 


Lessons Learned 

Relying on the convenience sample would have given 
a very inaccurate description of the public that was 
really of interest to MDC—all Missouri riders. This is 
not to say convenience samples are always wrong. 
Sometimes we may only need to know the opinions of 
the most avid or engaged constituents. But it is 
important to remember that a convenience sample, 
whether comments at a public meeting or a formal 
questionnaire, only represents the views of those 
individuals who answered and should not be expanded 
to “the public”. Whether you are considering 
conducting your own survey about a local issue or 
reading a report with survey results, make sure you 
understand the sample frame! 
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